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Abstract 


This paper calls for new approaches to the evaluation of physical learning environments 
and suggests future directions for research in the field. Prior research across the primary, 
secondary and tertiary education sectors has revealed a variety of metrics by which the 
performance of educational facilities can be measured. However, if physical learning 
environments are to be considered as spaces that provide a range of affordances for 
teaching and learning, improved methods are required to evaluate the effectiveness of ‘units 
of the environment’ as pedagogical settings. 


For more than 40 years, the field of post-occupancy evaluation has provided direction 
regarding how evidence can be gathered about the performance of educational facilities in 
use, yet such work has mostly overlooked the evaluation of learning spaces for pedagogical 
effectiveness. Research being conducted as part of the Evaluating 21st Century Learning 
Environments (E21LE) ARC Linkage project indicates that a return to the origins of post- 
occupancy evaluation in the field of environmental psychology is required in order to 
support the development of more rigorous methods of learning environment evaluation — 
methods that can provide valuable feedback about how effective ‘units of the environment’ 
are for their intended purpose, supporting teaching and learning. 


Introduction 


What are commonly termed ‘new generation learning environments’ (NGLEs) — defined here as 
learning spaces that provide a greater degree of spatial variation, geographic freedom and access to 
resources for students and teachers than traditional classrooms — are becoming common-place in 
Australian schools. The proliferation of these ‘non-traditional’ learning spaces has become a 
phenomenon as new facilities are built and existing facilities are refurbished (Saltmarsh et al, 2014). 
However, not all NGLEs are equal: a variety of spaces tend to get grouped under this umbrella. In the 
primary and secondary school sector Dovey and Fisher (2014) identified at least five distinguishable 
building typologies that could be considered NGLEs. These range from classrooms that have been 
updated with contemporary furniture and digital technologies, to transformable spaces that can be 
opened-up or closed-down through the use of sliding panels, to large open spaces that commonly 
feature interior elements that help situate diverse teaching and learning activities. 


The variety of ‘new’ facility typologies being built in Australia, and internationally (OECD, 2013), 
has provided a rich testing ground for updating pedagogical practice, both in individual schools and 
across whole school systems. With such an opportunity comes the need to evaluate these learning 
environments to determine which are best supporting desired teaching and learning practices, activities 
and behaviours. 


The conclusions of a journal article co-authored by this author in 2014 provide the ‘launching point’ 
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for this paper. Based on a critical review of the literature on the ‘evaluation of physical learning 
environments’ Cleveland and Fisher (2014) formed the following conclusions: 


1) Approaches to evaluations that attempt to assess the effectiveness of physical learning 
environments in supporting pedagogical activities are in their infancy and require further 
development; 

2) More studies are required in order to develop rigorous methodologies and methods that can be 
confidently employed to assess the effectiveness of physical learning environments in 
supporting desired teaching and learning practices, activities and behaviours; 

3) Such research could profit from an interdisciplinary approach that involves people from a 
variety of backgrounds, including but not limited to education, human geography, 
environmental psychology and architecture; and 

4) The development of formative evaluation methodologies, which could support the evaluation 
of educational facilities throughout their lifecycle, appears to be warranted (p. 24-5). 


Following-up on these conclusions, this paper provides a suggested framework for the research 
suggested above. In doing so, it discusses some of the work currently being undertaken by members of 
the Learning Environments Applied Research Network (LEaRN) at the University of Melbourne 
through an ARC Linkage project titled, Evaluating 21” century learning environments (E21LE). This 
research is being conducted based on the contention that if physical learning environments are to be 
considered as spaces that provide a range of affordances for teaching and learning, improved methods 
are required to evaluate the effectiveness of units of the environment (Barker, 1968) as pedagogical 
settings. 


To support arguments put forward later in the paper for evaluation methodologies and methods that 
can take into account the influence of units of the environment on the experiences of teachers and 
students (i.e. pedagogical practice), the paper provides a brief literature review that covers a range of 
issues concerning the field of evaluation and then goes on to discuss the domains of environmental 
psychology and critical human geography. The literature reviewed (1) explores selected theories that 
have informed the field of evaluation, and some pragmatic issues that evaluators should address when 
setting-up, conducting and reporting on evaluations, and (2) identifies the relevance of the domains of 
environmental psychology and critical human geography to learning environment evaluation. 


Subsequently, after briefly discussing the metrics by which the ‘performance’ of educational facilities 
have been measured in the past, the paper suggests future directions for research and evaluation. For 
more than 40 years, the field of post-occupancy evaluation (POE) has provided direction regarding 
how evidence can be gathered about the performance of educational facilities in use, yet such work 
has mostly overlooked the evaluation of learning spaces for pedagogical effectiveness. Research being 
conducted as part of the E21LE project indicates that a return to the origins of post-occupancy 
evaluation in the field of environmental psychology is required in order to support the development of 
more rigorous methods of learning environment evaluation — methods that can provide valuable 
feedback about how effective units of the environment are for their intended purpose, supporting 
teaching and learning. 


Literature review 
Evaluation: Approaches to assessing value and supporting decision making 


Defining evaluation: an evolving concept and tradition 


Evaluation is understood in various ways by various people. Hgjlund, (2014, p. 28) suggested that 
“evaluation is commonly understood as a tool informing policy-makers and civil servants of what 
works and what does not”. However, evaluation has been defined in alternative ways by various 
experts, revealing the board scope of the concept and some differing perspectives on what evaluation 
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is and what ‘work’ it can do. 


Scriven (1991) defined evaluation as the systematic determination of the quality or value of 
something; Cousins et al. (2004) defined evaluation as a process of systematic inquiry leading to 
judgements about the merit, worth and significance of a program or organisation; Davison (2004, p. 
85) defined evaluation as the “application of values to descriptive data so as to say something explicit 
about the quality or value of the evaluand in a particular context”; and Johnson et al. (2009, p. 378) 
defined evaluation as “any application of evaluation processes, products, or findings to produce an 
effect”. 


The literature indicates that the general logic of evaluation is inherently realist and rational and 
associated with assumptions about rationality and causality (Hg@jlund, 2014). Carman (2011, p. 351) 
commented that such logic tends to “place a high value on the rational, objective, and technical aspects 
of evaluation, with a considerable focus on using evaluation to make decisions” — a perspective firmly 
grounded in rational choice theory. However, Hgjlund (2014) suggested that the literature shows that 
evaluations rarely do change policies and that this constitutes a paradox, “since the very objective of 
evaluation is to improve policy” (p. 26). Commenting further, Hojlund (2014) proposed that, “ideally, 
evaluation improves policy through the instrumental application of an evaluation’s results 
(conclusions and recommendations)” (p. 29), yet went on to suggest that “positivist assumptions 
behind evaluation have been weakened somewhat over the last decades, as positivism [has been] 
challenged by phenomenological and hermeneutic traditions as well as critical theory” (p. 29), thus 
highlighting a trend towards more relational, rather than rational, framings of evaluation. 


Evaluating: what, why and how 


Davison (2004) suggested that evaluations are generally conducted for two purposes: (1) to find areas 
for improvement; and/or (2) to generate an assessment of overall quality or value for reporting or 
decision-making purposes. Davison (2004) identified the following things as commonly evaluated: 


Projects, programs or organisations 

Personnel or performance 

Policies or strategies 

Products or services 

Processes or systems 

Proposals, contract bids or job applications (p. 1). 


In framing evaluations, Davison (2004) suggested that the purpose of an evaluation should be 
carefully considered. If evaluating for accountability, she suggested that it was best to have an 
independent evaluation, but if the goal was more focussed on organisational capacity building or 
learning, she concluded that it was important to include stakeholder participation (a more relational 
framing of evaluation). 


Evaluation theories may describe and prescribe what evaluators do, or should do, when conducting 
evaluations (Coryn et al., 2011). Coryn et al. suggested that evaluation theory helps to guide people’s 
choices about “evaluation purposes, users, and uses, who participates in the evaluation process and to 
what extent, general activities or strategies, method choices, and roles and responsibilities of the 
evaluator” (p. 199). To improve the likelihood of a good fit between an evaluation and its 
environment, Chelimsky (2013) concluded that people involved in setting-up evaluations (i.e. 
evaluators) should also consider the following: 


1) The kind of evaluation that may be feasible, based on what has been learned about the 
program context and especially its history; 

2) The types of evaluation questions that will be possible to answer; and 

3) The methods—individual or combined—that are appropriate (p. 94). 
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To this end, Carman (2011, p. 368) commented that it is important to understand “why an organization 
chooses to engage in evaluation and how it intends to use the information”. She suggested that such 
information can “help evaluators to make important decisions relating to evaluation design, data 
collection, and measurement”. Further-to-this, Chelimsky (2013) counselled that as evaluations are 
performed in the real world, they are open to “political pressures by policy makers, planners, 
administrators, special interest groups, subject-area practitioners, participants, and all those who may 
be affected by the results—or feared results—of the evaluation” (p. 92). She suggested that such 
influences should be recognized and accounted for in appropriate ways and recommended that for an 
evaluation to be viable, the design must: 


1) Examine contextual factors; 

2) Set up a plan for dealing with potential problems of credibility and use; 

3) Lay a foundation for predicting and tracking the key external factors likely to affect the 
evaluation from beginning to end (p. 94). 


With regard to reporting on evaluations, Chelimsky (2013) identified that it is important to produce a 
report that is technically accurate, but is also clearly written and without jargon. She suggested that 
reporting should not simply take the form of written reports and that face-to-face briefings and 
presentations to those involved in the evaluation, and especially those in a position to affect use, 
should be conducted to enable the processes, products and findings of an evaluation to be 
appropriately shared. 


Finally, Davison (2004) concluded that it is important to take the time to critically review the quality 
of an evaluation itself. She suggested that evaluations should be judged on the following criteria: (1) 
the validity of their conclusions, (2) their utility to relevant stakeholders, (3) the way in which they 
were conducted, (4) credibility, and (5) cost. 


Environmental psychology, critical human geography and education 


Weinstein (1981) proposed that physical environments can have an impact on learning by moderating 
social, psychological and instructional variables. Based on findings from environmental psychology 
studies into person-environment relations, she suggested that the physical spaces in schools can 
facilitate or inhibit learning through both ‘direct effects’, such as noise or crowding, and through 
‘symbolic effects’, such as when poor conditions communicate to students a lack of respect for them 
on the part of the school they attend. For these reasons, Weinstein recommended that learning 
environments in schools should be considered as important as the curriculum and that the physical 
aspects of learning should be carefully planned by teachers in order to match with teaching objectives 
and the learning needs of students. 


Weinstein’s perspective, derived from the traditions of environmental psychology (e.g. Barker & 
Gump, 1964; Barker, 1968; 1976), provides a foundation for the arguments put forward in the later 
parts of this paper with respect to re-framing of methods used to evaluate physical learning 
environments (see ‘Renewed approaches to the evaluation of physical learning environments’). To 
appropriately situate these arguments in the literature, this section of the paper provides some 
important background on the fields of both critical human geography and environmental psychology. 


Soja’s critical human geography 


Soja (1989) suggested that only recently (from the 1980s) has the interpretive significance of space 
been recognised within the realms of critical social theory and given rise to the discipline of ‘critical 
human geography’. It was his contention that the influence of space should be considered more 
rigorously when seeking understandings of the social world. He proposed that critical human 
geography opened up avenues for the interpretation of social histories and settings through a critical 
spatialization. Such an approach, he claimed, may complement the temporal or sequential time-based 
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histories that have historically been central to critical social theory. Soja (1989, p. 25) promoted his 
approach to understanding ‘space-time-being’ as follows: 


Just as space, time, and matter delineate and encompass the essential qualities of the 
physical world, spatiality, temporality, and the social being can be seen as the abstract 
dimensions which together comprise all facets of human existence. ... How this ontological 
nexus of space-time-being is conceptually specified and given particular meaning in the 
explanation of concrete events and occurrences is the generative source of all social theory, 
critical or otherwise. 


Relating these ideas about ‘space, time, being’ to schools, Fisher (2002) argued that: 


Critical human geography is another of the disciplines ... [that is required] in order to make 
some theoretical sense of why schools, as political places and spaces, are seemingly 
unconscious of the power of space (p. 167). 


Lived experience of space: the value of the user perspective 


Building on Soja’s ideas, Lees (2001) recommended a critical ethnographic approach to drawing out 
meaning within the context of a critical geography of architecture. She suggested that in order to gain 
understandings of architecture from a critical geographic perspective we must go beyond trying to 
understand architecture from a representational viewpoint and investigate the ways spaces are socially 
produced. 


Providing a backdrop for these ideas, Lefebvre (1991a) argued that the production of space is never 
innocent and championed the spatial perspectives of inhabitants, or users, of different spaces. It was 
his contention that inhabitants felt space more than thought about it and therefore they encountered a 
concrete or subjective view of space through their lived experience (Merrifield, 2000). Furthermore, 
he suggested that the lived experience of space went beyond the visual to become experienced through 
all the senses. 


Lefebvre juxtaposed these ideas about the lived experience of space with the ways in which he 
suggested architects and planners often experience and encounter space. It was his contention that 
architects and planners often operate within, and experience space in, the abstract of the design phase. 
Believing that user experience of space should be more closely considered when trying to understand 
the significance of space, he commented that, “what we are concerned with here is not texts (blue 
prints) but texture” (Lefebvre, 1991b, p. 138). In supporting Lefebvre’s ideas about how to approach 
an understanding of architectural design, Merrifield (2000) stressed the importance of thinking about 
whose space we mean. 


Behaviour settings theory 


Behaviour settings theory was established by Barker (1968; 1976), an ecological/environmental 
psychologist, and his colleagues (Barker & Gump, 1964; Gump, 1974, 1980; Schoggen, 1989) to 
explain the influences that ‘units of the environment’ (behaviour settings) have on human behaviour. 
Together, they demonstrated that recognisable units of the environment have a powerful influence 
over the ways people behave. Indeed, they found that behaviour settings often had a stronger influence 
on people’s behaviour than a person’s individual inclinations. Scott (2005) explained this further. He 
suggested that behaviour settings coerce people to conduct themselves in certain ways as they 
encountered particular settings. 


Behaviour settings theory recognises physical and social components of each unit of the environment. 
The physical components, or milieu, are characterised by a specific set of time, place and object props, 
and the social components are characterised by a set of attached ‘standing patterns of behaviour’ 
(Barker, 1968). Thus, behaviour settings are composed of a variety of interior entities and events, 
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including people, objects (e.g. chairs, walls, pens, paper, computers), behaviour (e.g. lecturing, 
listening, sitting), and other processes (e.g. air circulation, sound transfer) (Barker, 1976). These 
components of the environment form patterns that form the boundaries of a behaviour setting and 
distinguish one setting from another. Bechtel (1977, p. 33) described the boundary of a behaviour 
setting as, “the place where the behaviour stops”. Gump (1980) further stratified the physical 
components of behaviour settings to isolate the physical milieu from the human components. He 
identified three major components of behaviour settings: milieu, human components and program. 


Heft concluded that although behaviour settings exist independently of individuals, they “occur 
naturally as a function of the collective actions of a group of individuals” (2001, p. 253). As a result, 
behaviour settings do not change as individuals enter or exit, as long as an adequate number of 
individuals remain. To this end, Barker (1976, p. 19) suggested that “it is common observation that the 
same people and objects are transformed into different patterns as they pass from one variety of setting 
to another’. Indeed, both the physical and social components of a behaviour setting must be present 
for the setting to exist (Scott, 2005). For example, a game of cricket may be recognised as a behaviour 
setting. For this behaviour setting to exist, a sporting field and the required equipment must be present 
along with the required behaviours of the players. Should any one of these components be absent then 
the behaviour setting would cease to function. 


Bechtel (1977) and Schoggen (1989) suggested that behaviour settings are often bounded by 
architecture. Both contended that architectural space can play a significant role in establishing 
behaviour settings by determining the physical boundaries of behaviour settings. Further to this, 
Bechtel (1977, p. vii) suggested that behaviour settings theory could provide a platform from which to 
pursue social goals through architectural design. 


Gump (1974) promoted behaviour settings theory as a useful theoretical lens through which to 
investigate the role of space in schools, suggesting that: 


Education is an environmental enterprise. Some have thought that it could be advanced by 
reliance on learning theory or principles from child development, but these thoughts arose 
out of social science's inability to deal with environments. As an environmental enterprise, 
education requires knowledge about environmental "milieu-with-program" units and 
concepts (Gump, 1974, p. 593). 


Gump (1980) and others (e.g. Gislason, 2010) have since used behaviour settings theory to inform 
their thinking about education as an environmental enterprise. 


Past practices in learning environment evaluation 


Metrics by which the performance of school facilities have been measured 


In recent decades, approaches to learning environment evaluation have largely focused on features of 
the physical environment itself. For example, Sanoff’s (2001) School Building Rating Scale tool is 
organized around the assessment of the following variables: 


Physical features 

Outdoor areas 

Learning environments 

Social areas 

Media access 

Transition spaces and circulation routes 
Visual appearance 

Degree of safety and security 
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e Overall impression 
e Personal information 


In contrast, but still with a strong focus on features of the physical environment, the Design Quality 
Indicators for Schools (DQIfS) tool developed in the UK (CABE, 2005) provides another example of a 
school evaluation approach organized around physical variables. These include the following: 


Functionality 

Access 

Space 

Uses 

Build quality 
Performance 
Engineering services 
Construction 

Impact 

School in its community 
Within the school 

Form and materials 
Character and innovation 


While all important measurement variables, both examples omit consideration of the social or human 
components of the learning environment. 


Early approaches to building evaluations 


The first systematic building evaluations were conducted during the 1960s by academic researchers 
with backgrounds in environmental psychology (Cooper, 2001). In keeping with the literature 
reviewed above, these groups were interested in the interaction of people and their environment and 
wished to ‘‘make building design more rigorous and systematic’’ (Cooper, 2001 p. 159). Some of the 
earliest evaluations were performed on university dormitories in the USA (Preiser and Nasar, 2008) 
and on a variety of non-domestic buildings in the UK (Cooper, 2001). 


Following many building evaluations being carried out on public works projects and government 
buildings in the UK, USA, Canada, New Zealand and Australia during the 1980s, a lack of funding, 
and a perception that the lens of environmental psychology had failed to deliver because of a 
disconnection between the process and conduct of evaluations and the use of evaluative findings, was 
blamed for the cessation of such activity (Preiser and Nasar, 2008). 


More recently, prominent figures in the field of POE and Building Performance Evaluation (i.e. 
Preiser and Nasar, 2008) have reported that academic researchers have again become interested in 
building evaluation and begun to develop new perspectives from which to consider such evaluations. 
They reported that: 


The 21st century has seen a new paradigm replacing the hierarchical, command and 
control, top-down approach with a consumer-oriented democratic approach, one that is 
autonomous, self-organizing, ecological, to sustain adaptation and continuous improvement 
... It calls for fairness, open, two-way communication, community building, cooperation, 
trust and honesty. For places experienced by the public (building exteriors, and interiors 
used by many people), the values of the public (the consumer) take priority (pp. 88-89). 


With respect to the higher education sector, the conclusions of the Learning Landscapes in Higher 
Education report (CERD, 2010) support this notion. This report concluded that evaluation should 
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move from ‘‘a focus on ‘spaces’ to ‘places’ with an emphasis on the social and pedagogic rather than 
the financial and the material (p. 47). 


Renewed approaches to the evaluation of physical learning environments 


Evaluating the effectiveness of units of the environment as pedagogical settings: 
Future directions for research and evaluation 


Although building evaluation methods that were informed by theories of environmental psychology 
may have fallen from grace during the period from the late-1980s to the mid-2000s, it would appear 
that such frameworks are again in their ascendency. Addressing the missing link between (1) the 
findings of evaluations that make connections between peoples’ lived experiences of units of the 
environment (i.e. their responses to different behaviour settings) and (2) the use of such findings, 
appears to be the loci of potential improvement in the way school facilities can be evaluated and 
understood. Gaining people’s confidence in evaluation methodologies and methods that make strong 
connections between ‘pedagogy and space’ would appear to be a key step forward. 


In agreement with Ornstien et al’s (2009) conclusion that ‘‘user-informed assessments increase the 
likelihood that a given school building fulfils its intended educational purposes to the greatest degree 
possible’’ (p. 364), it would appear logical that if learning environments are to be assessed for the 
ways they can support desired teaching and learning practices, activities and behaviours, they must be 
assessed subjectively within the context of the educational model(s) they are intended to support. 
Approaching school learning environment evaluations in such a way would overcome some well 
identified gaps in the literature. For example, Pearshouse et al (2009, p. 4) identified ‘‘a need for the 
educational sector as a whole to reconsider how to evaluate physical learning spaces, so as to more 
clearly assess how they satisfy design intentions and teaching and learning needs”, while Gislason 
(2010) posited that: 


Few studies of any kind have linked school design with the human interactions that govern 
learning environments, and none drew substantive conclusions about how the use and 
configuration of instructional space frame teaching and learning (p. 128). 


Evaluative frameworks that were aligned with Gislason’s (2010) model for school design research! 
would address connections between the physical and social components of units of the environment. 
Gislason’s model highlighted: 

1) Ecology — building design, technology and other material elements; 

2) Organization — teaching, scheduling and curriculum; 

3) Student milieu — learning and motivation, social climate; and 

4) Staff culture — assumptions, values, and patterns of thought and behavior. 


Evaluating school learning environments through the lens’ of critical human geography and 
environmental psychology would not only provide important information about the design of learning 
spaces but also critically about how such environments were inhabited and used by teachers and 
students (i.e. information about the ‘programs’, ‘processes’ and ‘systems’ operating within and in 
connection to learning spaces). Evaluative findings about the later could aid the development of what 
Saltmarch et al (2014) described as ‘spatially responsive pedagogies’, which they suggested are 
underpinned by “commitment to collective learning with, about and within a particular environment” 
(p. 12). Involving teachers in the processes and outcomes of learning environment evaluations could 
also assist school leaders to effect a range of pedagogically-oriented changes in their schools. As 
Hargreaves and Fullan (2012) suggested, leadership for transformative change in teaching involves a 
mixture of ‘push, pull, and nudge’ effects. Sponsorship and participation in learning environment 


1 After Owens and Valesky (2007) and by deduction after Barker and Gump (1964). 
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evaluations that produce findings linking pedagogical activity with the affordances of units of the 
environment could not only provide important information about what is working (or not) with respect 
to the design of learning environments, but also with respect to the pedagogical inhabitation and use of 
such environments. 


Conclusion 


Based on the ideas discussed above about the value of critical human geography and environmental 
psychology as lens’ for the evaluation of units of the environment in schools, the E21LE project is 
aiming to provide a range of evaluative strategies and tools that can be utilized by schools and 
governing agencies to influence decisions about (1) what types of learning spaces should be built or 
refurbished, and (2) how school communities can get the most out of the spaces they already have 
through professional capacity building (improvement). Via both strategies, the project hopes to inform 
the push, pull and nudge factors of pedagogical change. 
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